Some computing systems (e.g. graphics processing systems) include graphics rendering logic configured to receive graphics data (e.g. from an application running on a computer system) and render the graphics data to provide a rendering output. For example, the graphics data provided to a graphics processing system may describe geometry within a three dimensional (3D) scene to be rendered, and the rendering output may be a rendered image of the scene.
Some graphics processing systems (which may be referred to as “tile-based” graphics processing systems) use a rendering space which is subdivided into a plurality of tiles. The “tiles” are regions of the rendering space, and may have any suitable shape, but are typically rectangular (where the term “rectangular” includes square). To give some examples, a tile may cover a 16×16 block of pixels or a 32×32 block of pixels of an image to be rendered. Therefore, a tile of the rendering space corresponds to a region of the rendered image. As is known in the art, there are many benefits to subdividing the rendering space into tiles. For example, subdividing the rendering space into tiles allows an image to be rendered in a tile-by-tile manner, wherein graphics data for a tile can be temporarily stored “on-chip” during the rendering of the tile.
Tile-based graphics processing systems typically operate in two phases: a geometry processing phase and an image rendering phase. In the geometry processing phase, the graphics data for a render is analysed to determine, for each of the tiles, which graphics data items are present within that tile. Then in the image rendering phase, a tile can be rendered by processing those graphics data items which are determined to be present within that tile (without needing to process graphics data items which were determined in the geometry processing phase to not be present within the particular tile). The graphics data items may represent geometric shapes, which describe surfaces of structures in the scene, and which are referred to as “primitives”. A common primitive shape is a triangle, but primitives may be other 2D shapes or may be lines or points also. Objects can be composed of one or more (e.g. hundreds, thousands or millions) of such primitives.
In some computing systems, a rendered image which is rendered by graphics rendering logic may be subsequently processed by image processing logic. In this way, post-processing can be performed on the rendered image. For example, the image processing logic may apply video encoding to the rendered image to encode the image. Applying video encoding to a rendered image may reduce the amount of data used to represent the image. This can be particularly useful if the image is to be transmitted across a network (e.g. over the internet or a local network, e.g. using WiFi) or if the image is to be stored. As an example, an image is may represent a frame within a sequence of frames which are to be rendered and encoded at a computing system and then transmitted to a display for display thereon in real-time. Video encoding methods are known in the art, which typically conform to a video encoding standard. There are many different video encoding standards, such as the H.265, H.264, VP8, VP9, DivX video standards, and many more. An aim of encoding an image is often to reduce the amount of data that is used to represent the image. Therefore, the encoding of the image may involve some compression of the image. The compression may be lossless or lossy.
There is often redundancy in the initial image data which represents the values of each of the pixels in the image, because there is often some correlation between pixel values in the image, e.g. for pixels which are located next to, or at least near to, each other in the image. For example, the colour of a group of pixels which all have the same colour can be compressed (losslessly) by storing the colour once along with some grouping information to represent the colour of all of the pixels of the group rather than by storing the colour multiple times, i.e. once for each of the pixels. The more redundancy there is in the image, the more the image can be compressed without losing any of the information in the image. A quantization parameter (QP) can be used by the encoder to regulate how much detail is saved in the image. For example, when the QP used by the encoder is very small, almost all the detail in the image is retained. When the QP used by the encoder is increased, some of the detail in the image is aggregated so that the bit rate of the encoded image drops, but at the price of some increase in distortion and some loss of quality of the encoded image.
It may be beneficial for an encoder to provide an encoded image at a constant (or near constant) bitrate both across the image, and when the image is a frame within a video stream comprising multiple frames across the different frames of the video stream. A buffer may be used, and operated as a ‘leaky bucket’, which is emptied at a constant rate whilst being filled at a variable rate with encoded image data from the video encoder. A rate controller may be used to adjust the QP used by the video encoder in accordance with a target bitrate and such that the buffer is not depleted or saturated. The rate controller may receive feedback from the is video encoder to indicate the number of bits that have been generated by encoding previous images and/or previous sections of an image currently being encoded. The rate controller may use the feedback to determine how the QP should be varied for encoding subsequent images and/or subsequent sections of an image currently being encoded.
The redundancy in an image may vary significantly from image to image as well as from one section of an image to another section of that same image. It may be that the image can be encoded only once (e.g. when the image is encoded in real-time), such that an image is not re-encoded if too few or too many bits are generated, relative to the target bitrate, by encoding the image. Furthermore, in order to reduce the storage requirements of the buffer and to keep the latency of the encoding of the image to an acceptably low level, the size of the buffer is constrained. Furthermore, larger variations in the QP used to encode different sections of the same image may result in a perceptually lower quality encoded image. Therefore, the rate controller might not allow the QP to vary by more than a threshold amount for the encoding of an image.
The constraints mentioned above can result in cases in which the rate controller does not optimally (or even nearly optimally) control the QP used by the video encoder to encode the sections of an image (e.g. when the image is encoded in real-time). For example, the amount of detail in an image may be localised, e.g. the bottom quarter of the image may include more detail than the rest of the image. In this situation, the rate controller may be operating well within a bit budget for three quarters of the image, using up three quarters of the bit budget. However, when the video encoder encodes the bottom quarter of the image containing a high level of detail, the rate controller is forced to increase the QP (compared to that which would be well suited for encoding the level of detail in the bottom quarter of the image) in order for the video encoder to stay within the bit budget. Changing the QP in this way may result in a perceptually low quality encoded image.